2025-06-10
Warning
Difference-in-differences requires some form of parallel trends: similar time trends for treated and untreated units.
Question
What is the effect of these policies on cigarette sales and vaccine uptake, respectively?
Generally requires:
One (or few) treated units
Many untreated units
Long pre-treatment history of outcomes for all units
Post-treatment outcomes for time period of interest
Find a weighted average of the control units that best approximates the pre-treatment history of the treated unit(s).
This is the synthetic control unit, which is then compared to the treated unit’s outcomes in the post-treatment period.
\[ \hat{\theta}_t = Y_{1t} - \sum_{i=1}^n w_i Y_{0it}, \]
where \(Y_{1t}\) is the outcome of the treated unit in period \(t\) and \(Y_{0it}\) is the outcome of the \(i\)th untreated unit in period \(t\).
The weights \(w_1, \ldots, w_n\) are nonnegative and sum to 1.
In the simplest form, the weights are chosen to minimize:
\[ \sum_{t=1}^{T-1} \left( Y_{1t} - \sum_{i=1}^n w_i Y_{0it} \right)^2, \]
where \(T-1\) is the last period for which the treated unit is pre-intervention.
Covariates can be incorporated in the weight minimization. For covariates (including pre-treatment outcomes) labelled \(k=1,\ldots,K\), choose a weight vector \(w\) that minimizes:
\[ \sum_{k=1}^K v_k \left( X_{1k} - \left(\sum_{i=1}^n X_{0ik} w_i \right) \right)^2, \]
where \(v_k\) are weights on the importance of each covariate, which can themselves be chosen to minimize the pre-intervention difference or by cross-validation on a split sample of pre-intervention times.
Important
The estimand is again the average treatment effect on the treated (ATT): the effect of the policy on the treated unit compared to if it had not been treated.
The choice of time and scale for the comparison can be made by the investigator based on subject-matter knowledge.
The pre-intervention mean squared prediction error (MSPE) of the SC fit can be used to assess fit.
To test robustness of results, can change:
Control units
Time frame considered
Covariates used
Conduct the SC analysis with the same specifications for each control unit, excluding the treated unit. This gives a null distribution of estimates.
Can exclude those with much higher pre-intervention MSPEs.
Visual inspection of the observed result compared to the null distribution can follow. Or a specific estimator can be used to conduct a hypothesis test.
Common choices are:
Can also run the analysis on dummy intervention time points.
Note
This is similar to the cross-validation approach sometimes used to select covariate weights.
If using both, interpret with caution.
In some cases, non-affected outcomes or populations may be available. These can be used as a null control or distribution.
Advantages:
Allows non-parallel trends
Interpretability of weights
Counterfactual estimate can be used for many ATT estimands
Disadvantages/Limitations:
Requires linear interpolation of trends
Lots of researcher degrees of freedom
Can be highly variable or sensitive to specifications
Clearest with one or few treated units
Ensure ATT is appropriate
Consider trade-offs: more vs. fewer units, more vs. fewer time periods, interpolation vs. extrapolation, etc.
Pre-specify analyses: control units, covariates, years, placebo tests, MSPE restrictions, etc.
Run robustness checks wherever possible
Interpret results in context